Robust Structure from Motion in the Presence of Outliers and Missing Data

The paper preposes a new scheme to promote the robustness of 3D structure and motion factorization from uncalibrated image sequences. First, an augmented affine factorization algorithm is proposed to circumvent the difficulty in image registration with imperfect data. Then, an alternative weighted factorization algorithm is designed to handle the missing data and measurement uncertainties in the tracking matrix. Finally, a robust structure and motion factorization scheme is proposed to deal with outlying and missing data. The novelty and main contribution of the paper are as follows: (i) The augmented factorization algorithm is a new addition to previous affine factorization family for both rigid and nonrigid objects; (ii) it is demonstrated that image reprojection residuals are in general proportional to the error magnitude in the tracking data, and thus, the outliers can be detected directly from the distribution of image reprojection residuals, which are then used to estimate the uncertainty of inlying measurement; (iii) the robust factorization scheme is proved empirically to be more efficient and more accurate than other robust algorithms; and (iv) the proposed approach can be directly applied to nonrigid scenarios. Extensive experiments on synthetic data and real images demonstrate the advantages of the proposed approach.